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2. 

Abstract 

Although the Miller Analogies Test (MAT) is used extensively as an 
admissions test for graduate programs, relatively little research 
has examined the validity of the MAT for predicting subsequent 
graduate student achievement and no published studies have examined 
differential prediction of student performance from MAT scores. 
There were two purposes of this study. The first purpose was to 
investigate the validity of the MAT for predicting the achievement 
of graduate students in education. The second purpose of this 
study was to examine the extent of age and gender differences in 
prediction of graduate student performance from MAT scores. The 
students included in this analysis were a sample of 1,438 students 
who began master ' s-level study in education at a large private 
university during a seven-year period. Results from this study 
indicated that MAT scores were significantly correlated with 
students' cumulative graduate grade point average and with their 
grade performance in five specific graduate courses. These results 
also indicated that MAT scores do not predict graduate performance 
similarly for male and female students and for students of all ages. 
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One facet of educational measurement that is of interest to 
professionals in testing, admissions, and institutional research 
is the utility of tests used for admissions purposes. For under- 
graduate admissions, the two most commonly used tests are the 
American College Testing Program (ACT) and the Scholastic Aptitude 
Test, which will soon be renamed the Scholastic Assessment Tests 
(SAT). At the graduate level, the Graduate Record Examination 
(GRE) and the Miller Analogies Test (MAT) are widely used. Although 
several previous studies have evaluated the validity of these tests 
as predictors of subsequent student achievement, it has been recomm- 
ended that institutions conduct additional research on the predictive 
validity of admissions tests (Whitney, 1989). These validity studies 
should investigate the relationship between admissions test scores 
and several criterion measures of student achievement such as 
cumulative grade point average (GPA) , grades in specific courses, 
or degree attainment (Hartnett & Willingham, 1980). Despite a 
large number of previous validity studies, however, considerably 
fewer studies have investigated whether these tests predict the 
performance of all subgroups of students similarly (Drasgow, 1987). 

Although the MAT is used extensively as an admissions test for 
graduate programs, relatively little research has examined the 
validity of the MAT for predicting subsequent graduate student 
achievement. The small number of existing studies have been 
characterized by small sample sizes and have produced conflicting 
results. In fact, no previous studies have investigated whether 
MAT scores predict subsequent achievement similarly for all sub- 
groups of graduate students. In the following discussion, prior 
research on the predictive validity of the MAT will be reviewed 
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and an introduction to differential prediction and prediction bias 
will be presented. Previous findings on age and gender bias in 
prediction of student achievement from ACT, SAT, and GRE scores 
will be reviewed. It will become evident that further study on 
the predictive validity of the MAT is needed and that research on 
prediction bias from MAT scores should be conducted. 

Predictive Validity of the MAT 

A limited number of studies have evaluated the predictive 
validity of the MAT and those studies have yielded conflicting 
results. The criterion measure that has been most commonly used 
in MAT validity studies is cumulative graduate grade point average 
(GPA). Several studies have found MAT scores to be significant 
predictors of subsequent grade performance. For example, Furst 
and Roelfs (1979) found a significant correlation between MAT 
scores and the graduate GPA of 348 students in a doctoral program 
in education. Similarly, Cantwell (1990) found that MAT scores 
were significantly related to the graduate grade performance of 81 
students in a counselor education program. Graham (1991) noted a 
significant relationship between MAT scores and the graduate grades 
of 50 students in a Master of Business Administration program. 
Finally, MAT scores were significant predictors of graduate GPA for 
a sample of 73 students in a master ' s-level psychology program 
(Littlepage, Bragg, & Rust, 1978). However, a number of studies 
have failed to find a significant relationship between MAT scores 
and graduate GPA. Williams, Harlow, and Gab (1970) evaluated the 
records of 84 doctoral students in education and found a slightly 
negative correlation (-.03) between MAT scores and students' 



cumulative graduate GPA. Similarly, Pederici and Schuerger (1974) 
failed to find a significant relationship between MAT scores and 
the graduate grades of students in psychology while Vacc and Picot 
(1984) also found that MAT scores were not significant predictors 
of the graduate grades of students in education. 

A smaller number of studies have assessed the predictive 
validity of the MAT for other types of academic outcomes and the 
results have tended to find MAT scores to be significant predictors 
of later performance. Two studies have examined students 1 grades 
in specific courses as the criterion measure. Mehrabian (1969) 
found that MAT scores were significantly correlated with grades 
in specific psychology courses and with statistics grades for 
graduate students in psychology. Furst and Roelfs (1979) also 
found that MAT scores were significant predictors of grades in 
statistics for a sample of graduate students in education. In 
addition, Kirnan and Geisinger (1981) found that MAT scores were 
significantly correlated with the performance of graduate students 
in psychology on their master's comprehensive examinations. Yet 
another criterion measure that has been studied is degree completion 
Three studies have evaluated the predictive validity of the MAT when 
using the criterion measure of students 1 completion vs. non-complet- 
ion of their graduate degrees. None of the three studies found MAT 
scores to be significant predictors of whether or not students 
completed their graduate degrees in education (Nagi, 1975; Williams, 
Harlow, & Gab, 1970) or in psychology (Rawls, Rawls, & Harrison, 
1969). ^ 

Differential Validity and Prediction Bias 

A great deal of research has been directed to the definition 
and identification of test bias and several approaches have been 
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proposed. Most interest has been focused on the differential 
performance of various groups of students, particularly minority 
students and women (Cole & Moss, 1989; Moore, 1989). There have 
been numerous studies which have found that minority students 
attained lower mean scores on standardized tests than majority 
students (Moore, 1989? Stanley, 1971). Similarly, several studies 
have found that female students scored lower on specific types of 
tests, particularly tests of mathematics achievement and specific 
types of mathematics test items, than did male students (Bridgeman & 
Wendler, 1991; Doolittle & Cleary, 1987). In addition, considerable 
attention has been given to the effects of differential test scores 
on the selection of students for particular programs (such as 
graduate programs) or on the placement of students into remedial 
programs (Bickel, Hammel, & O'Connell, 1975; O'Connor, 1989; Mehrens, 
1989). An important consideration in the use of standardized tests 
for selection purposes is that test scores predict subsequent 
achievement similarly for all groups of students who take the test. 
When systematic differences exist in the relationship between the 
test score and the criterion measure for various subgroups of 
students, then prediction bias exists (Kaplan, 1985; Reynolds, 
1982). Prediction bias refers to a systematic error in predicting 
the criterion variable for particular groups of students. For 
example, it is important to determine if MAT scores predict the 
subsequent achievement of graduate students in education similarly 
for all subgroups of students. In this case, the presence of 
significant differences between subgroups of students, such as 
male and female students or younger and older sfudents, in how 
accurately achievement was predicted from MAT scores would provide 
evidence of prediction bias. 
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Two methods have been proposed for the study of bias in pred- 
iction of student achievement from admissions test scores. The 
first approach involves the comparison of validity coefficients 
for all subgroups of students, and is sometimes referred to as an 
examination of differential validity (Jensen, 1980). For example, 
the correlations between MAT scores and graduate GPA would be 
computed separately for male and female students and those correla- 
tions would then be tested for a significant difference. Similarly, 
age differences in predictive validity would be investigated by 
computing the validity coefficients for younger and older students 
and then testing those correlations for a significant difference. 
Specific statistical tests are available to test for a significant 
difference between two correlation coefficients (Kleinbaum & Kupper, 
1978) or for a significant difference between three or more correl- 
ations (Edwards, 1984) and those methods have been applied in 
previous research on differential validity (House & Johnson, 1992, 
1993b) . 

A second method used to investigate prediction bias involves 
the comparison of subgroups of students on the mean error of 
prediction of GPA from admissions test scores (Reynolds,. 1982). 
This approach involves several steps. First, a regression equation 
based on the entire sample of students is used to compute a predicted 
GPA for each student. A residual score (the predicted GPA minus the 
actual GPA) is then computed for each individual. The mean error 
for each group of students (gender group or age group) is the mean 
of the residual scores for each individual in the group (Pedhazur, 
1982). For example, the mean error of prediction for male students 
would be the mean of the residual scores of each male student. 
Finally, subgroups of students are compared using analysis of 
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variance (ANOVA) to test for significant differences in residual 
scores, which represents systematic error in the prediction of GPA 
from admissions test scores* This approach for investigating 
prediction bias was first developed by Dr. T.A. Cleary and is some- 
times referred to as the Cleary model (Hulin, Drasgow, & Parsons, 
1983). 

Previous Research on Age Bias 

In most testing situations, tests are designed for use with 
younger traditional-aged students. As a result, relatively little 
is known about the predictive validity of admissions tests for older 
nontraditional students (Strieker & Rock, 1987). In addition, only 
a limited number of studies have investigated age differences in the 
prediction of grade performance from admissions test scores. Results 
from a study of graduate students in several disciplines indicated 
that older female students tended to earn graduate grade point 
averages that were higher than predicted by their GRE scores while 
younger female students earned graduate grade point averages that 
were somewhat lower than predicted by their GRE scores (Swinton, 
1987). These results provided evidence that GRE scores did not 
predict subsequent academic performance similarly for students of 
all ages. However, Swinton (1987) did not evaluate students in 
graduate programs in education. House (1989) investigated age bias 
in prediction of graduate grades from GRE scores for a large sample 
of graduate students in education and obtained results similar to 
those of Swinton (1987); GRE-Quant itat ive and GRE-Total scores 
(GRE-Verbal scores plus GRE-Quant itat ive scores) underpredicted 
the performance of older students and overpredicted the graduate 
grade point average of younger students. It should be noted, 
however, that despite the presence of age bias in prediction of 
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grade performance from GRE scores, there was no evidence of 
differential prediction of achievement from GRE scores. That 
is, no significant differences were found between younger and 
older students for the correlations between GRE scores and later 
graduate grade point average (House, 1989). 

Evidence for age bias in prediction of student achievement 
from admissions test scores has also been reported in a cross- 
cultural setting. Zeidner (1987b) examined the relationship between 
SAT scores and academic achievement for students at a university in 
Israel. One finding was that the correlation between SAT scores 
and subsequent GPA was significantly lower for students over 30 
years of age than for younger students. Zeidner (1987b) also found 
age bias in the prediction of subsequent grade performance from 
SAT scores. Finally, age bias in prediction of student grade point 
average from GRE scores has been reported for a second sample of 
graduate students in education (Matthews & Martin, 1992). 

The findings of these studies indicate that admissions test 
scores may not predict subsequent GPA similarly for students of 
all age groups. Student age is a variable that should be considered 
in future validity studies (Matthews & Martin, 1992). It should 
also be noted that none of the previous age bias studies examined 
the MAT as the admissions test. Consequently, further study is 
needed to determine if similar results would be obtained from the 
prediction of graduate GPA from MAT scores. 

Previous Research on Gender Bias 

Several studies have investigated gender bias in prediction of 
student grade performance from admissions test scores. In general, 
the results of these studies have indicated that admissions test 
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scores tend to underpredict the academic performance of female 
students and to overpredict the achievement of male students. That 
is, female students tend to earn grade point averages that are higher 
than were predicted from their test scores while male students tend 
to earn lower grades than were predicted from their admissions test 
scores. Research on undergraduate students has found this pattern 
of underpredict ion of female students' GPA from both ACT and SAT 
scores. Considering students at several universities, Linn (1973) 
found a systematic underpredict ion of female students' GPA from SAT 
scores. Breland and Griswold (1982) also found that female students' 
grade point averages were underestimated by ACT scores while Zeidner 
(1987a) provided similar evidence of gender bias in grade performance 
from SAT scores for students in a cross-cultural setting. Finally, 
Sawyer (1986) examined gender bias in prediction of first-year 
college grades from ACT scores using data from over 100,000 students 
at over 200 colleges. As was the case with findings from the SAT, 
Sawyer (1986) found that ACT scores underpredicted the subsequent 
grade performance of female students and overpredicted the first- 
year college grades of male students. 

One factor that may influence gender bias in prediction of 
grade performance from admissions test scores is that male and 
female students may enroll in different types of courses that might 
have differing grading standards. Previous research has suggested 
that there are considerable differences between major fields in the 
grading standards that are used (Goldman, Schmidt, Hewitt, & Fisher, 
1974; Strenta & Elliot, 1987) while other studies have shown that 
the predictive relationship between admissions test scores and grade 
performance can differ substantially within distinct program areas in 
in the same discipline (House & Johnson, 1993a; Kirnan fc Geisinger, 
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1981). Consequently , research that considered the effects of diff- 
erent majors for male and female students has provided additional 
insight into the extent of gender bias. Gamache and Novick (1985) 
attempted to control for the effects of differential grading by 
separating students by their major fields: Business, Liberal Arts, 
Pre-Medicine , and Undecided, Their findings indicated that differ- 
ential prediction was evident within programs and that, as was found 
in earlier research, ACT scores underpredicted the actual grade 
performance of female students and overpredicted the grade point 
average of male students. Finally f McCornack and McLeod (1988) 
controlled for the effects of differing grading standards by examin- 
ing course grades as the criterion measure. In contrast to previous 
findings, McCornack and McLeod (1988) failed to find evidence of 
gender bias in the prediction of college course grades from SAT 
scores. Consequently, the appearance of gender bias in the 
prediction of GPA from admissions test scores may be due to the 
effects of differential grading practices on the criterion measure 
of cumulative GPA. 

There has been little research on gender bias in the prediction 
of graduate student performance from GRE scores. Kaczmarek and 
Franco (1986) found that GRE scores did not predict subsequent 
grade performance similarly for a small sample of male and female 
students in a master 1 s-level program in counseling. There have 
been no studies that have examined gender bias in the prediction 
of graduate student performance from MAT scores. Consequently, 
research is needed to investigate gender bias in prediction from 
MAT scores, and that research should consider the effects of differ- 
ential grading standards on gender bias. 
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Statement of Purpose 

There were three main purposes of this study. First, this 
study was designed to investigate the validity of the MAT for the 
prediction of subsequent graduate student performance. Because 
previous research has indicated that the predictive validity of 
an admissions test can vary substantially for different types of 
criterion measures (Hartnett & Willingham, 1980; House, Johnson, & 
Tolone, 1987), several criterion measures of academic performance 
were examined. The second purpose of this study was to investigate 
the extent of gender bias in the prediction of student achievement 
from MAT scores. Considerable research has assessed gender bias 
in the prediction of undergraduate students' grade performance from 
ACT and SAT scores. This study was designed to build upon those 
studies by examining gender bias in the prediction of graduate 
student achievement. Finally, the third purpose of this study 
was to examine the extent of age bias in the prediction of graduate 
student performance from MAT scores. Previous research on age bias 
has been conducted using GRE scores as predictors (House, 1989; 
Swinton, 1987), and this study was designed to extent those findings. 

Methods 

Students 

The students included in this analysis were a sample of 1,438 
students (mean age « 32.7 years, SD « 8.00) who began master's- 
level study in education at a private university during a seven- 
year period. In this sample, there were 192 male students and 1,246 
female students. In the entire sample, there 75 African-American 
students, 15 Asian students, 29 Hispanic students, and 1,306 white 
students; there were 13 students in this sample whose ethnic status 

o 13 
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was identified as other or not given. For analyses done by age 
group, there were 261 students who were 24 years of age or younger 
when they began graduate study and there were l f 177 students who 
■were 25 years of age or older when they began graduate study. 
Measures 

Demographic data collected for each student included their 
gender, ethnic group, and age at the time they began graduate study. 
Each students 1 MAT score was also collected. Finally, data for 
eight criterion measures of academic performance were also compiled 
for each student. Two measures related to degree attainment were 
collected: whether or not students completed their master's degree 
and the length of time (in years) required for the completion of 
the degree. Finally, five measures of grade performance were also 
collected: students' cumulative graduate grade point average and 
their grades in five courses required of all graduate students in 
education (Survey of Child Development, Theories of Teaching and 
Learning, Introduction to Graduate Research, Research for Teachers, 
and ^istory and Philosophy of Education). 
Procedures 

The data from this study were analyzed in several ways. First, 
descriptive statistics for MAT scores and each measure of grade 
performance were computed. This was done for the entire sample 
and separately for male and female students and by student age 
group. In order to be consistent with earlier research of this 
type, younger students were defined as 24 years of age and younger 
at the time graduate study was initiated while older students were 
those who were 25 years of age and older. 

In order to investigate the predictive validity of the MAT, 
Pearson product-moment validity coefficients were computed for the 
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MAT and each criterion measure. For a criterion measure that is 
binary (graduated vs. not-graduated) f the point-biserial correlation 
is ordinarily recommended. However, the Pearson correlation coeffi- 
cient provides a satisfactory estimate of the true correlation 
between a binary variable and a continuous variable (Leonard, 1976). 
These correlation coefficients between MAT scores and each criterion 
measure were computed for the entire sample and separately for male 
and female students. Correlations were also computed separately for 
younger and older students. The correlation coefficients for male 
and female students and for younger and older students were then 
tested for equivalence using a Z-transf ormation procedure (Kleinbaum 
& Kupper , 1978) . 

The method used to determine prediction bias was to compare 
students by age group and by gender group on the mean error of 
prediction found for the MAT. A regression equation based on 
all students was used to compute a predicted GPA for each student. 
A residual score (the predicted GPA minus the actual GPA) was then 
computed for each individual. The mean error for each group was 
the mean of the residual scores, and the groups were then compared 
using analysis of variance (ANOVA) to test for systematic error in 
the prediction of graduate grades from MAT scores. This procedure 
was done separately for cumulative graduate GPA and for grades in 
each of the five required graduate courses. 

Results 

Descriptive Statistics 

Descriptive statistics for MAT scores, the length of time 
required for master's degree completion, and each measure of grade 
performance for the entire sample are presented in Table 1. The 
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descriptive statistics for male and female students are also 
summarized in Table 1. Preliminary tests indicated that, for this 
sample, male students had significantly higher MAT scores (t(1436) = 
2.17, p = .0301) while female students had significantly higher 
cumulative graduate grade point averages (t (1436)=5.57, p=.0001). 
Female students also earned significantly higer grades than did 
male students in two of the five specific courses that were examined. 
In each case, the initial tests for homogeneity of variances indic- 
ated that male and female students had significantly different 
variances for their course grades; consequently, a t-test procedure 
for unequal variances was used (Leonard, 1976). Female students 



Table 1 

Descriptive Statistics for Predictor Variables and Criterion 
Measures (For All Students and by Student Gender) 





All 




Males 


Females 


Variable 


M 


SD 


M 


SD 


M 


SD 


Miller Analogies Test 


47.67 


15.12 


50.02 


16.33 


47.31 


16.07 


Cumulative Graduate 
Grade Point Average 


3.83 


0.24 


3.74 


0.35 


3.85 


0.22 


Time to Degree (Years) 


2.82 


1.48 


2.56 


1.32 


2.86 


1.50 


Survey of Child 
Development 


3.84 


0.39 


3.70 


0.55 


3.86 


0.36 


Theories of 

Teaching and Learning 


3.77 


0.46 


3.68 


0.55 


3.78 


0.44 


Introduction to 
Graduate Research 


3.80 


0.43 


3.72 


0.53 


3.82 


0.41 


Research for Teachers 


3.88 


0.35 


3.84 


0.38 


3.88 


0.34 


History & Philosophy 
of Education 


3.78 


0.46 


3.73 


0.53 


3.78 


0.45 
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earned significantly higher grades in Survey of Child Development 
(t(10Q.9) = 2.75, p = .0007) and in Theories of Teaching and Learning 
(t(158.9) = 2.04, p = .0428). 

Descriptive statistics summarized by student age group are 
presented in Table 2. Preliminary tests indicated that older 
students had significantly higher MAT scores than did younger 
students (t(446) = 6.74, p = .0001). Older students had signific- 
antly higher grades in Theories of Teaching and Learning (t(244.7) = 
3.10, p = .0022) and in Research for Teachers (t(255.4) = 3.62, p = 
.0004). The analyses for each course used the t-test procedure 
for unequal variances. 



Table 2 

Descriptive Statistics for Predictor Variables and Criterion 
Measures (By Student Age Group) 



Younger Older 
Variable M SD M SD 



Miller Analogies Test 42.34 13.56 48.85 16.41 
Cumulative Graduate 

Grade Point Average 3.81 0.21 3.84 0.25 

Time to Degree (Years) 2.78 1.55 2.84 1.47 
Survey of Child 

Development 3.83 0.39 3.84 0.39 
Theories of 

Teaching and Learning 3.66 0.52 3.80 0.44 
Introduction to 

Graduate Research 3.80 0.44 3.81 0.42 

Research for Teachers 3.78 0.43 3.90 0.33 
History & Philosophy 

of Education 3.77 0.45 3.78 0.46 
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Predictive Validity of the MAT 

Correlations between MAT scores and each criterion measure of 
academic performance for the entire sample are summarize I in Table 
3. MAT scores were significant predictors of students' cumulative 
graduate GPA and of their grade performance in each of the five 
specific courses. However, MAT scores were not significantly 
correlated with whether or not students had completed their master's 
degrees or with the length of time (in years) required to complete 
a master's degree in education. The relationship between MAT scores 
and graduate student acheivement was also examined separately for 
male and female students and those results are also presented in 
Table 3. When considering male students, the correlations that were 
found to be significant were the same as was the case for the entire 

Table 3 

Correlations Between MAT Scores and Criterion Measures 
(For All Students and By Student Gender) 



Outcome Measure 


All 
Students 


Male 
Students 


Female 
Students 


Z 




Cumulative Graduate 
Grade Point Average 


.273** 


.255** 


.298** 


0. 


62 


Graduated/Not Graduated 


.013 


-.075 


.027 


1. 


34 


Time to Degree Completion 


.057 


.020 


.065* 


0. 


50 


Survey of Child Development 


.188** 


.325** 


.180** 


1. 


40 


Theories of Teaching & Learning 


.241** 


.354** 


.223** 


1. 


51 


Intro to Graduate Research 


.256** 


.352** 


.247** 


1. 


21 


Research for Teachers 


.167** 


.244** 


.156** 


1. 


02 


History & Philosophy of Educ. 


.282** 


.252** 


.290** 


0. 


42 



**£ < .01, *p_ < .05. 
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sample; for male students, MAT scores were significantly correlated 
with subsequent cumulative graduate GPA and with grades in each of 
the five specific courses. For female students, MAT scores were 
significantly related to seven of the eight criterion measures; only 
the correlation between MAT scores and degree completion was not 
significant. Finally, no significant differences between male and 
female students were noted for any of the relationships between MAT 
scores and the academic performance measures included in this study. 
The lack of any significant differences indicates that the correla- 
tions found for male and female students can be considered estimates 
of the same population values. 

Correlations between MAT scores and each measure of academic 
performance were analyzed separately for each student age group and 
those findings are presented in Table 4. For younger students, MAT 



Table 4 

Correlations Between MAT Scores and Criterion Measures 
(By Student Age Group) 



Outcome Measure 


Younger 
(Less Than 24) 


Older 
(25+) 


Z 


Cumulative Graduate 
Grade Point Average 


.274** 


.269** 


0.07 


Graduated/Not Graduated 


.010 


.020 


0.14 


Time to Degree Completion 


.020 


.063 


0.56 


Survey of Child Development 


.237** 


.176** 


0.75 


Theories of Teaching & Learning 


.210** 


.234** 


0.28 


Intro to Graduate Research 


.199** 


.276** 


1.14 


Research for Teachers 


.097 


.163** 


0.88 


History & Philosophy of Educ. 


.273** 


.290** 


0.25 


**p_ < .01, *p_ < .05. 


13 







19. 

scores were significant predictors of cumulative graduate GPA and 
of grade performance in four of the specific courses. For older 
students f MAT scores were significant predictors of cumulative 
graduate GPA and of grade performance in all five of the specific 
courses. The relationship between MAT scores and subsequent degree 
completion was not significant for either group of students. Simil- 
arly, the relationship between MAT scores and the length of time 
(in years) required for students to complete their master's degree 
was not significant for either group of students. Finally, no 
significant differences between younger and older students were 
found for any of the correlations between MAT scores and any of 
of the eight measures of academic performance. 
Prediction Bias 

Findings from the analyses of gender bias in prediction of each 
measure of grade performance are summarized in Table 5 and several 
Table 5 

Gender Differences in Prediction of Grade Performance From 
Miller Analogies Test Scores 





Mean 


Error 








Outcome 


Males 


Females 


F 


df 


P 


Cumulative Graduate 
Grade Point Average 


.099 


-.015 


41.39 


1,1436 


.0001 


Survey of Child 
Development 


.160 


-.020 


18.66 


1,820 


.0001 


Theories of Teaching 
and Learning 


.103 


-.019 


8.07 


1,825 


.0046 


Introduction to 
Graduate Research 


.099 


-.014 


8.85 


1,1085 


.0030 


Research for Teachers 


.042 


-.007 


2.41 


1,989 


.1209 


History & Philosophy 
of Education 


.067 


-.009 


3.05 


1,965 


.0809 
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significant results were obtained. First, a significant difference 
between male and female students for the mean error of prediction 
of cumulative graduate GPA from MAT scores was found. In this case, 
MAT scores significantly underpredicted the cumulative graduate 
grade point average of female students and overpredicted the cumul- 
ative graduate GPA of male students. This finding indicates that 
female students earned higher cumulative graduate grade point aver- 
ages than were predicted from their MAT scores while male students 
earned lower cumulative graduate grade point averages than were 
predicted by their MAT scores. Similar findings were also noted for 
three of the five specific graduate courses. Significant differences 
between male and female students for the mean error of prediction of 
grade performance in those three courses were found. For those three 
courses (Survey of Child Development, Theories of Teaching and 
Learning, and Introduction to Graduate Research), female students 
earned grades that were higher than were predicted by their MAT 
scores while male students earned grades that were lower than were 
predicted by their MAT scores. Finally, although significant 
differences between male and female students for the mean error of 
prediction of grade performance in the remaining two courses (Res- 
earch for Teachers and History and Philosophy of Education) were 
not found, the same pattern of underpredict ion of female grade per- 
formance from MAT scores was observed. 

Findings from the analyses of age bias in prediction of grade 
performance from MAT scores are presented in Table 6. There was not 
a significant difference between younger and older students for the 
mean error of prediction of cumulative graduate grade point average 
from MAT scores. This indicates that MAT scores predicted graduate 
GPA equally accurately for younger and older students. Significant 
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differences between younger and older students for the mean error 
of prediction of grade performance in two of the five specific 
courses wfere found. For those two courses (Theories of Teaching and 
Learning and Research for Teachers), MAT scores underpredicted the 
grades earned by older students and overpredicted the grades earned 
by younger students. This finding indicates that, for those two 
courses, older students earned grades that were higher than were 
predicted by their MAT scores while younger students earned grades 
that were lower than were predicted by their MAT scores. Finally, 
for the three courses for which there were not significant differ- 
ences in the mean error of prediction, MAT scores tended to under- 
predict the grade performance of younger students and overpredict 
the grades earned by older students. 

Table 6 

Age Differences in Prediction of Grade Performance From 
Miller Analogies Test Scores 



Mean Error 

Outcome Younger Older F df p 

Cumulative Graduate 

Grade Point Average -.001 .000 0.01 1,1436 .9188 
Survey of Child 

Development -.015 .004 0.38 1,820 .5355 

Theories of Teaching 

and Learning .071 -.019 5.67 1,825 .0175 

Introduction to 

Graduate Research -.039 .010 2.57 1,1085 .1094 

Research for Teachers .073 -.018 10.95 1,989 .0010 
History and Philosophy 

of Education -.045 .012 2.67 1,965 .1023 
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Discussion 

There were three main findings from this study* First, MAT 
scores were found to be significant predictors of cumulative 
graduate GPA and of grade performance in five specific graduate 
courses. These findings are consistent with the results of prev- 
ious studies that have found significant correlations between MAT 
scores and cumulative graduate GPA (Cantwell, 1990; Furst & Roelfs r 
1979; Graham, 1991) and between MAT scores and grades in specific 
graduate courses {Furst & Roelfs, 1979; Mehrabian, 1969). There 
was not a significant relationship between MAT scores and students* 
degree completion status in this study. This lack of a significant 
predictive relationship between MAT scores and degree completion is 
also similar to the findings of previous research (Nagi, 1975; 
Rawls f Rawls, & Harrison, 1969; Williams, Harlow, & Gab, 1970). 

The second finding from this study was that MAT scores did 
not predict graduate grade performance similarly for male and female 
students. For cumulative graduate GPA and for grades in three of 
the five graduate courses examined, MAT scores underpredicted the 
achievement of female students and overpredicted the achievement of 
male students. The results for cumulative graduate GPA are similar 
to previous results from undergraduate .admissions tests (Linn, 1973; 
Sawyer, 1986; Zeidner, 1987). However, because it has been suggested 
that gender bias should not be apparent for the prediction of grades 
in specific courses (McCornack & McLeod, 1988), the findings of 
gender bias for the grades in specific courses indicate that factors 
other than simply differential grading standards are involved in the 
appearance of gender bias. 

The third finding from this study was that MAT scores did not 
consistently predict graduate grade performance in a similar manner 



for younger and older students. There was not a significant 
difference between younger and older students in the mean error 
of prediction of cumulative graduate GPA. This lack of significance 
differs from previous research that found age bias in the prediction 
of cumulative graduate GPA from GRE scores (House, 1989), However, 
for two of the five graduate courses examined in this study, MAT 
scores underpredicted the grades earned by older students and over- 
predicted the grades earned by younger students. The pattern of 
older students earning higher grades than predicted by their test 
scores is similar to findings from previous studies that used 
cumulative graduate GPA as the criterion measure (House, 1989; 
Swinton, 1987; Zeidner, 1987b). 

There are some limitations to the present study. First, only 
students at one institution were included in the study. Further 
research is needed to determine if these results would be found for 
students at other types of institutions. Previous multi-institut- 
ional validity studies have been conducted. Rubin (1980) conducted 
an analysis of the predictive validity of the Law School Aptitude 
Test (LSAT) at 82 law schools while Zwick (1993) evaluated the 
predictive validity of the Graduate Management Admission Test (GMAT) 
for students in doctoral program in business and management at 36 
universities. Similar research is needed to assess the predictive 
validity of the MAT for graduate students in education at various 
types of institutions. A second limitation of the present study is 
that no analysis of the effects of restriction of range were made. 
It has been shown that restricting the distribution of admissions 
test scores for students who have been admitted for graduate study 
will produce lower than expected predictive validity (Givner & 

Hynes, 1979). Previous studies have investigated the effects of 
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restriction of range on the predictive- validity of the GRE in an 
actual admissions situation (House, 1983) and the predictive 
validity of the GRE when restriction of range was not a factor 
Huitema & Stein, 1993). Similar studies need to be done to assess 
the effects of restriction of range on the predictive validity of 
the MAT found in this study. Finally, based on the recommendations 
made by Sawyer (1987), there were insufficient numbers of minority 
students in this sample to allow meaningful analyses of ethnic bias 
in prediction of grade performance. Studies of ethnic bias have 
been conducted for the ACT (Maxey & Sawyer, 1981; Sawyer, 1986) and 
for the SAT (Cameron, 1989). Research is needed to determine if 
MAT scores predict subsequent grade performance similarly for grad- 
uate students of all ethnic groups. 

The results of this study suggest a number of directions for 
further research. First, although age and gender differences were 
found for the relationship between MAT scores and grade performance, 
no analysis was made of age or gender differences in performance on 
individual items of the MAT. A number of techniques have been 
developed to investigate item bias (Holland & Thayer, 1988; Ironson, 
1982; Thissen, Steinberg, & Wainer, 1988). Previous studies of 
gender differences in performance on individual tests items have 
been conducted for several admissions tests, including the ACT 
(Handrick & Loyd, 1982), the SAT (Becker, 1990; Carleton & Marco, 
1982), and the GRE (Donlan, Hicks, & Wallmark, 1980). Similar 
research is needed to determine if item bias would be found for the 
MAT. A second direction for further research would be to determine 
if the predictive validity of the MAT is similar for graduate 
students in a variety of disciplines. Previous research has shown 
that the predictive validity of the GRE can vary considerably for 
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different fields of graduate study (Thornell & McCoy, 1985). A 
final direction for further research would be to investigate the 
improvement in prediction of graduate student performance by comb- 
ining MAT scores with noncognitive measures. Previous research on 
undergraduate students has indicated that noncognitive variables 
are significant predictors of several types of academic outcomes 
such as grade performance in specific courses (Gordon, 1989; House, 
1993a) and of withdrawal from college (House, 1992b, 1993b). Recent 
research has indicated that, in some instances, noncognitive variab- 
les are more significant than admissions tests scores as predictors 
of student achievement (House, 1992a). Consequently, research is 
needed to determine if combining noncognitive measures vrith MAT 
scores would result in improved prediction of graduate student 
performance. 

These results indicate that, for some measures of graduate 
student achievement, the predictive validity of the MAT is signific- 
ant. The findings of this study also indicate that the MAT does 
not predict subsequent achievement similarly for male and female 
students or for students of all ages. Finally, these results provide 
directions for further research. 
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